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Abstract 

Let G be a linear algebraic group defined over an algebraically 
closed field. The double coset question addressed in this paper is the 
following: Given closed subgroups X and P, is X\G/P finite or in- 
finite? We limit ourselves to the case where X is maximal rank and 
reductive and P parabolic. This paper presents a criterion for in- 
finiteness which involves only dimensions of centralizers of semisimple 
elements. This result is then applied to finish the classification of those 
X which are spherical. Finally, excluding a case in F±, we show that 
if X\G/P is finite then X is spherical or the Levi factor of P is spher- 
ical. This implies that it is rare for X\G/P to be finite. The primary 
method is to descend to calculations at the finite group level and then 
to use elementary character theory. 



Keywords: algebraic groups, finite groups of Lie type, double cosets, 
spherical subgroups, finite orbit modules. 

1 Introduction 

All algebraic groups in this paper are linear algebraic groups defined over an 
algebraically closed field and all subgroups are assumed to be closed. Given 
an algebraic group G we wish to classify those subgroups X and P such that 
X\G/P is finite. One has that X\G/P is finite if and only if the G-orbit 
G/P splits into finitely many X-orbits. This viewpoint makes a complete 
classification of all finite double coset collections appear unlikely in the near 
future. For this reason, we will restrict which subgroups we allow. We will 
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generally assume G is a reductive (or simple) algebraic group, that P is a 
parabolic subgroup and that X is maximal rank and reductive. This paper is 
concerned with proving that a double coset collection is infinite. We intend 
to establish finiteness results in a later paper. 

We will state the main results of the paper first with brief indications of 
how these results relate to earlier work in the field. This is followed by a 
lengthier description of some of this earlier work. 

The first result provides a powerful criterion for establishing that X\G/P 
is infinite. If H is a group and g G H we write H g for the centralizer of g in 
H. 

Theorem 1.1 (Dimension Criterion). Let G be a reductive algebraic 
group, X and P subgroups of G with X maximal rank and P parabolic. 
Let L be a Levi factor of P and let s E X n L be a semisimple element. 
If dim Z(G S ) + dimCs > dimX, + dimP s (equivalently, if dim Z(G S ) + 
\ dim G s — dim X s — \ dim L s > 0), then X S \G S /P S andX\G/P are infinite. 

Classification of maximal rank reductive spherical subgroups. For 

the next result, suppose G is a simple algebraic group. The first applica- 
tion of the dimension criterion is to finish the classification of maximal rank 
reductive spherical subgroups. The subgroup X is spherical if a Borel 
subgroup B has a dense orbit upon X\G. Brion [5] and Vinberg [28] inde- 
pendently showed that X is spherical if and only if B has a finite number 
of orbits upon X\G, or, equivalently, X\G/B is finite. A maximal rank 
reductive subgroup is generic if a subgroup of the same type exists in all 
characteristics. The work of Kramer [17], Brundan [6] and Lawther [18] 
has produced a list of subgroups which are spherical in all characteristics. 
The generic maximal rank reductive subgroups on this list are given in table 
1 where we use the following conventions. We treat A$ and Bq as trivial 
groups and D\ as a 1-dimensional torus. We list only the Lie type of each 
group, as the property of being spherical is not affected by which represen- 
tative of an isogeny class is used. The notation Ti refers to an i dimensional 
torus, central in X. Finally, in the group A\A\ < G2 the tilde ~ is used 
to signify a subgroup with short roots (we don't use this notation for the 
other groups as there is no ambiguity). Finally, in attempting to classify 
the maximal rank reductive spherical subgroups, it suffices to classify only 
the generic cases as the others arise from isogenics or graph automorphisms, 
which preserve the property of being spherical (see Lemma 3.2). 

To prove that table 1 is complete, we introduce the following root- 
theoretic property which was inspired by Lawther's anti-open property (see 
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Table 1: Generic Maximal Rank Reductive Spherical Subgroups 



X < G 


X < G 


A n A rn T\ ^ -^n+m+l 


EqTi < Ej 


B n T) m 5~ ^n+m 


A 7 < E 7 


A n -lTi < B n 


AiD 6 < E 7 


C n C m < C n j rTn 


A ± E 7 < E 8 


C n -\T\ < C n 


D$ < Eg 


A n — \T\ < C n 


A 1 C 3 < F 4 




B 4 < F 4 


^ n _iT! < £> n 


A 2 < G 2 




A X A X < G 2 


AiA 5 < £ 6 





Theorem 2.2 below). Let X be a maximal rank reductive subgroup of G. We 
abbreviate the phrase "maximal rank reductive" with MRR. Fix a maximal 
torus of X and define $(X) and $(G) with respect to this maximal torus. 

We say X has an ^-complement if there exists a closed root subsystem 
R < 3>(G) — &{X). This is equivalent to the existence of a generic MRR 
subgroup K < G with &(K) = R and Kflla maximal torus. The adjective 
"long" or "short" may be applied if R has only long or short roots. 

Theorem 1.2. Let G be a simple algebraic group and X a generic MRR 
subgroup. The following are equivalent: 

(i) X is spherical. 

(ii) X appears in table 1. 

(hi) X has no A 2 or B 2 complement. 

In this paper we show that (i) ==>■ (iii) ==> (ii) (more precisely, we 
show -i (ii) ==> -i (iii) => -> (i)). The other implications are due to 
Brundan [6] and Lawther (see Theorem 2.2 below). 

Theorem 1.2 applies to groups acting on the full flag variety G/B, where 
B is a Borel subgroup. Using the dimension criterion, we now obtain more 
general infiniteness results where P is a parabolic. Since table 1 contains rel- 
atively few subgroups, the following theorem places great restrictions upon 
X and P for X\G/P to be finite. An end node parabolic is conjugate to 
a standard parabolic obtained by crossing off exactly one of the end nodes 
in the Dynkin diagram of G. 
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Theorem 1.3 (Spherical X or Spherical L). Let G be a simple algebraic 
group, X a MRR subgroup, P a parabolic subgroup with Levi factor L. If 
G = F4 suppose P is not an end node parabolic. If X\G/P is finite then X 
is spherical or L is spherical. 

The extra restrictions placed upon P when G = F4 are necessary. In a 
later paper we will show that L^F^/P^ and L^F^/Pi are finite (where Pi 
is conjugate to the standard parabolic obtained by crossing off the i th node 
of the Dynkin diagram of F4, and Li is its Levi factor). 

Corollary 1.4. If X\G/P is finite and P is not maximal then X is spher- 
ical. 

Remark 1.5. The theorem and the corollary give a surprisingly strong 
dichotomy for MRR subgroups with respect to the double coset problem. 
Either they are spherical, or they will have an infinite number of orbits 
on almost all flag varieties. For instance, A\A^ is spherical in Eq, but 
X = T\A<~, will have an infinite number of orbits on all flag varieties Eq/P 
except, possibly, if P is an end node parabolic. As another example, suppose 
one could show that a MRR subgroup X in GL(V) has a finite number of 
orbits on flags consisting of one and two dimensional subspaces. Then X 
has a finite number of orbits on full flags, i.e. upon G/B where B is a Borcl 
subgroup. 

2 Some History of the Problem 

Spherical subgroups. The following result is a starting point for classi- 
fying reductive spherical subgroups. Let G be an algebraic group and let 
C g (t) be the centralizer of the involution r G Aut(G). 

Theorem 2.1 (Matsuki [20], Springer [23]). LetG be a reductive group. 
If the characteristic of the underlying field is not 2, then the centralizer of 
an involution in G is spherical. 

This result was proven by Matsuki in characteristic and extended to all 
characteristics except p = 2 by Springer. Seitz [22] has given an alternative 
proof and Lawther [18] has given a proof for positive characteristics which 
also shows that the same subgroups are spherical in characteristic 2. (Note, 
by "same" subgroups, we mean, by abuse of language, subgroups of the 
same type.) We will have occasion later to refer to Lawther's version of this 
theorem. In the following theorem let a±, . . . , a n be the simple roots of 
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the simple algebraic group G. Write the high root of G as Yl AjOj. Recall 
that a generic maximal rank reductive subgroup of G can be produced via 
the Borel-de Siebenthal algorithm [4]. This algorithm takes the Dynkin 
diagram, or the extended Dynkin diagram, of G, crosses off some number 
of nodes, and recursively applies the same procedure to the new diagrams 
which have been produced. 

Theorem 2.2 (Lawther [18]). Let G be a simple algebraic group with high 
root ^2 \&i is just described. Let X be a generic maximal rank reductive 
subgroup. The following conditions are equivalent and define anti-open. If 
X is anti-open then it is spherical. 

(i) There do not exist a, [3, a + /3 G <J>(G) - $(X). 

(ii) X can be produced from the Borel-de Siebenthal algorithm by removing 
a single node aj with Aj = 1 or by extending the diagram for G once 
and removing a single node oti with Aj = 2. 

(hi) If the characteristic of the underlying field is not 2, then X is the 

centralizer of an involution. 
(iv) (G,X) appears in table 1 but (G, X) {(B n , A n _iTi) ; (C„,C n _iTi), 

(G 2 ,A 2 )}. 

Remark 2.3. If X is anti-open then X has no A 2 or B 2 complement. The- 
orem 2.2 and Theorem 1.2 show that the converse is often, but not always, 
true. To be precise, the converse is true unless (G, X) G {(B n , A n -iT±), 
(C n ,C n -iTi), (G 2 , A 2 )}. For example, let G = B 2 , fix a maximal torus T 
and label the Dynkin diagram of G with a\ and a 2 where a.\ is long. The 
group X, generated by T and the root groups corresponding to the high 
root and its negative, is of type A\T\. Then a±, a 2 , a\ + a 2 G 3>(G) — &(X) 
but cci + 2a 2 G ®(X). So X does not have an A 2 or B 2 complement, but it 
is not anti-open. 

If the characteristic p is not 2, then C G (r), the centralizer of an invo- 
lution, forms a reductive group. The reductive spherical subgroups of the 
simple algebraic groups have been classified in characteristic by Kramer 
[17] and in fact most of them are centralizers of involutions. Brundan [6] gave 
a reduction mod p argument to show that certain groups in Kramer's list 
are spherical in all characteristics, but this argument could not be applied 
to nine of the cases. These nine exceptions were all groups which would be 
the centralizer of an involution when p was not 2. Lawther [18] introduced 
the concept of anti-open to prove that these remaining cases were spherical 
when p = 2. He then proceeded to work at the level of finite groups using 
character theory. See sections 4 and 5 below for some similar arguments. 
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In the work just described, the attention is focused on the group X. 
Related work has been done where the attention is focused on the action of 
a Borel subgroup B on varieties or modules. For example, the homogeneous 
space C g (t)\G is a symmetric variety, and these have been studied by a 
wide variety of authors in various contexts of representation theory. See, for 
example, Helminck [11] for how to compute the orbits of B upon C g (t)\G 
and for a review of related literature. In other contexts, the spherical sub- 
groups are of interest because they are related to multiplicity free modules 
(see the discussion below of work by Kac [16] and Arzhantsev [1]). 

Irreducible finite orbit modules. The next family of examples is of a 
representation theoretic nature. Let X be a closed, connected subgroup of 
G = GL(V) with V a finite dimensional vector space, irreducible under X. 
Then X is a reductive group and we may write X = X'Z where X' and Z 
are the derived subgroup and the center of X respectively. Then Z acts as 
scalars upon V. To make the following statement easier, we assume that 
Z equals the full group of scalars. Let P\ be the stabilizer in GL(V) of a 
1-space in V. We identify GL(V)/Pi with the collection of 1-spaces in V. 

Theorem 2.4. Let X, GL(V) and P\ be as described. Then X has a finite 
number of orbits upon V if and only if X'\GL(V) / P\ is finite. Moreover, 
all instances where X'\GL(V)/Pi is finite have been classified. 

The classification was found first by Kac [16] in characteristic and 
extended to positive characteristics by Guralnick, Liebeck, Macpherson and 
Seitz [12]. The latter paper also classifies when X'\GL(V) / P{ is finite where 
Pi is the stabilizer of an z-dimensional subspace. In Kac's paper, he sug- 
gests a broader context for this problem. Essentially, he wants to find those 
representations which allow a classification of orbits and he lists various 
properties which such representations might have in common. For instance, 
one of the properties he studies is that the modules be spherical (i.e. a Borel 
subgroup has a dense orbit on V) which is equivalent to the representation 
being multiplicity free. A related problem was studied by Arzhantsev [1]. 
Here the goal was to classify those representations of G on V such that a 
Borel subgroup has an open orbit in Gv for each v € V. Again, a classifi- 
cation is obtained. This demonstrates Kac's heuristic that representations 
with certain nice properties should be amenable to classification. The ques- 
tion remains as to which properties are important. The present paper takes 
a small step towards making the case that having a finite double coset col- 
lection is an important property. 
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A program for double cosets. The last two families of examples share 
a theme. In each case we have reductive groups acting on G/P where P is 
a parabolic subgroup. In an effort to classify all instances where X\G/Y 
is finite this setting is a natural place to concentrate for reasons we will 
discuss. Firstly, one should start by classifying finiteness of X\G/Y where 
both X and Y are maximal. Then the Borel-Tits Theorem (see [14, 30.4]) 
implies that each of X and Y is reductive or parabolic. If X and Y are 
both parabolics, then the collection is finite by the Bruhat decomposition. 
By work of Brundan [7] , if X and Y are each reductive and either maximal 
or the Levi factor of a parabolic subgroup, then finiteness of X\G/Y is 
equivalent to having a factorization G = XY. Such factorizations are rare 
and have been essentially classified in a paper by Liebeck, Saxl and Seitz [19]. 
(Note, having such a factorization is equivalent to having \X\G/Y\ = 1, so 
this is also an example of a finite double coset problem.) This leaves the 
case where X is reductive and Y = P is a parabolic. (Further discussion of 
these matters may be found in the article by Seitz [22].) 

Within the setting of X being reductive and P being parabolic, the 
question becomes which reductive groups should be addressed first. In many 
problems, the maximal rank reductive subgroups are the most important. 
Therefore in this paper we will study the case where X is maximal rank. 
We intend to classify all cases where X is maximal rank and reductive and 
X\G/P is finite in future papers. Afterwards, it is hoped that a classification 
may be obtained where the only restriction placed upon X will be that it is 
reductive. 

Outline of remaining sections. The outline of the rest of this paper 
is as follows: section 3 includes basic results and preliminaries; section 4 
reduces the double coset question of algebraic groups to a related question 
about finite groups; section 5 applies character theory to the finite groups 
(roughly following Lawther [18]) and obtains the Dimension Criterion; sec- 
tion 6 proves Theorems 1.2 and 1.3, assuming Proposition 6.4; section 7 
proves Proposition 6.4. 

3 Preliminaries 

In this section we list basic results which will be used later. Many (perhaps 
all) of the results in this section are known to others. We list them here either 
for convenience, or because references are difficult to find. For standard facts 
in algebraic groups we refer to the books by Borel [2], Humphreys [14] and 



INFINITENESS OF DOUBLE COSET COLLECTIONS 



8 



Springer [23]. 

For a reductive, connected reductive group, G we will use the following 
notation. Let T be a maximal torus, <I> be the root system and for each 
a G 3> let U a be the corresponding root group. 

Lemma 3.1. Let G be a simple algebraic group. All MRR subgroups of type 
A2, of the same length, are conjugate. All MRR subgroups of type B>2 are 
conjugate. If the rank of G is at least three then these subgroups are all Levi 
factors of parabolic subgroups. 

Proof. The last statement is clear. Let H and H' be two MRR subgroups 
which are claimed to be conjugate. By conjugation we may assume that H 
and H' share a common maximal torus T. If the rank of G is two then H and 
H' are equal. Otherwise H and H' are Levi factors and each is generated 
by T and the root groups (positive and negative) corresponding to a pair of 
adjacent nodes in the Dynkin diagram of G. Then H and H' are conjugate 
by the action of the Weyl group. □ 

The following lemma allows us to make a variety of easy reductions, or 
assumptions about G. 

Lemma 3.2. Let G be a group, X and P subgroups. Let Z be the center 
of G, suppose Z < P and let X, G and P be the images of X, G and P 
under the map G — > G/Z. Let K be a finite normal subgroup of G and let 
X, G and P be the images of X, G and P under the map G — > G/K. Let 
g,h € G. The following are equivalent: 



(i) 


X\G/P 


< 00. 


(") 


X\G/P 


< 00. 


(iii) 


X\G/P 


< 00. 


(iv) 


X9\G/P h \ < 00 



Proof. These statements can all be proven in an elementary fashion. □ 

Brundan [6] also states most of these. This lemma shows that the ques- 
tion of whether X\G/P is finite depends only upon the Lie type of the 
groups involved. In particular, it does not depend upon which elements of 
an isogeny class are chosen, the presence of centers, connectedness etc. By 
this result, we may work with either GL(V) or SL(F) and get essentially 
the same results. We may also assume that G has simply connected derived 
subgroup which eases some of the proofs. Finally, if X and P are maximal 
rank then we may assume that they contain a common maximal torus. 
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Convention. If r is an endomorphism we denote by G T the fixed points 
of r in G. If G is a group and g G G then G 9 denotes the centralizer of g in 
G. Finally, G Tt9 denotes those points in G fixed by both r and g. The finite 
groups of Lie type arise as the fixed points in G of a Frobenius morphism 
a : G — > G, where G is defined over the algebraic closure ¥ p of the field ¥ p 
of p elements. We refer to [8] and [26] for details. 

Lemma 3.3. Let G be a connected reductive group with simply connected 
derived subgroup. Let T be a maximal torus of G. 

(i) The center of G is contained in each maximal torus of G. (This does 
not require that G have simply connected derived subgroup.) 

(ii) If s G G is semisimple then G s is reductive and connected. 

(iii) For each s G T we have Z(G S ) < T. 

(iv) The set {G s \ s G T} is finite. Its size may be bounded by a constant 
depending only upon the root system of G. 

(v) Fix s G T. There exist ti, . . . ,t r G T such that {Gt \ t G G, Gt > 
G s } = {G u \ l<i<r}. Let Z(s) = {t G G \ G t = G s }. Then Z(s) 
is an open subset of Z{G S ) and its complement is \J i Z(Gt i ). 

(vi) If S is a torus and L = C G (S) then {s G S \ L = G s } is a dense subset 
ofS. 

Proof. Part (i) is [14, 26.2]. 
Part (ii) is [8, 3.5.4,3.5.6]. 

Part (iii). Note that T is a maximal torus of G s . By part (ii) we may 
apply part (i) to the group G s . 

Part (iv). By [8, 3.5.3] we have that G s is generated by T, the root 
groups it contains and by certain elements of the Weyl group. Since the 
Weyl group is finite and the number of root groups is finite, there are finitely 
many possibilities for G s . 

Part (v). Let U be the set proposed as the complement of Z(s). Clearly 
Gt > G s Z{Gt) < Z(G S ) < T. This, and the previous part, show 

that ti,...,t r may be chosen in T as stated and that U C Z(G S ). Since the 
union is finite, U is a closed set. Given t G Z(G S ) we have: 

t Z(s) G t > G s 

Gt = Gt t for some i 
<==> t G Z{Gt t ) for some i 
t G U. 

This shows that U is the desired complement. 
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Part (vi). Note that S < Z(L). Then for all t G S we have G t > L. By 
an argument similar to that for part (v), one can show that the set of t G S 
with Gt > L is a proper, closed subset of S. □ 

Lemma 3.4. Let G be a connected reductive group and a : G — ► G a 
Frobenius morphism. Then Z(G a ) = Z(G) a . Moreover, if G has simply 
connected derived subgroup and s,t G G a are semisimple elements, then 
G s = Gt •<=>■ G as = G a j- 

Proof. Carter [8, 3.6.8] shows that Z(G a ) = Z(G) a . For the second state- 
ment note that "=>" is obvious, for "<^" suppose G as = G a $. By Lemma 
3.3(h) we have that G s and Gt are connected and reductive. Then the first 
statement shows that Z(G ayS ) = Z(G s ) a whence t is in Z(G CT)S ) = Z{G S ) (T . 
This shows that t is in Z(G S ) whence Gt > G s . A symmetric argument 
shows that G s > Gt- □ 

Lemma 3.5 (Rational normalizer theorem). Let G be a connected re- 
ductive group defined over ¥ p , a : G — > G a Frobenius morphism. Let P be 
a a-stable parabolic subgroup. Then NG a (P a ) = Pa = (^G(P))a 

Proof. It is well known that P = Nq(P), which gives the second equality. 
For the first equality it is clear that P a < Ng (t {P ct ). The reverse inclusion 
follows from the fact that if P is a cr-stable parabolic subgroup with P a = P a 
then P = P, see [3, 4.20]. □ 

Corollary 3.6. Let G be connected and reductive, P a parabolic subgroup 
and x G G a . Suppose that a is a Frobenius morphism of G which fixes P and 
x. Let (G/P) x be the variety of G- conjugates of P which contain x. Then 
a acts upon (G/P) x and lpj(x) is equal to the number of a -fixed points on 
this variety. 

Proof. Using the Lang-Steinberg Theorem [25] it is easy to show that ip : 
G a /P a — > (G/P) a taking gP a to gP is an x-equivariant bijection. Together 
with the rational normalizer theorem this shows that we have bijections 
between (G/P) a , x , {G a /P a ) x and { 9 P a \ g G G a , x G 9 P a }- Elementary 
character theory shows that lp CT (x) equals the size of the last collection. 

□ 

Lemma 3.7 ([21, 3.5]). Let H be a connected algebraic group of dimension 
d and a : H — > H a standard q power Frobenius map. Then (q - l) d < 
\H a \ < (q+l) d . 



INFINITENESS OF DOUBLE COSET COLLECTIONS 



11 



Lemma 3.8. Let G be a connected reductive group with simply connected 
derived subgroup, a : G —> G a standard q th power Frobenius map and T a o~- 
stable maximal torus. Fix s £ T c and let Z(s) and t\, . . . ,t r be as in Lemma 
3.3. Let c\ and d2 be the number of components and dimension of Z(G S ) 
respectively. Let I C {1, . . . , r} such that dim Z(Gt i ) < dimZ{G s ) -4=^* i € 
/. Let m = \I\, let c 2 and d 2 be the maximal number of components and 
the greatest dimension of the Z(Gt i ) with i £ I. Note that if m ^ then 
d 2 < d\. 

Then Z(s) is a-stable and 

(q-l) dl -mc 2 {q + l) d * < \Z(s) a \ < Cl {q+l) d \ 

Proof. If t G Z{s) then G a M = o~{Gt) = o~G s = G s which shows that Z(s) is 
c-stable. We will be using Lemma 3.7 throughout the proof without further 
mention. 

Since Z(s) C Z(G S ) we see that \Z{s) a \ < \Z{G s ) a \ < c^q + l) dl where 
the second inequality is found by calculating \Z(G s ) a \ under the assumption 
that a stabilizes each component of Z(G S ). 

From Lemma 3.3 we have that Z(G S ) = Z(s) U \J i>1 Z(G ti ), where 
"U" indicates a disjoint union. Intersecting with sZ(G s )° and taking fixed 
points we have 

\(sZ(G s )° n Z(s)) a \ = \(sZ(G s )°) a \ - I (sZ(G s )° n (|J Z{G U ))) 

i>l 

It is easy to check that: 

\{sZ{G s yr\Z{s)) a \ < \Z(s) a \, 
(q-l) d i < \Z(G S )°\ = \(sZ(G s )°U 
(sZ(G s )°n({J ieI Z(G ti )))J\ < mc 2 (q + l) d 2. 

To finish we will prove that s^(G s ) n(Ui>i^(G ti )) = sZ(G s )°n(\J ieI Z(G U )). 
It suffices to show that sZ(G s )°nZ(G u ) is empty if dim Z(G U ) = dimZ(G s ). 
Let dim Z{G U ) = dimZ(G s ). Then Z{G U )° = Z{G S )° and sZ(G s )°r)Z(G ti ) 
is empty or all of sZ(G s )°. However, by definition of the ij, we have 
s Z(Gt i ) so we are done. □ 



4 Reduction to finite groups 

In this section we prove the fundamental results which relate double cosets 
in algebraic groups to double cosets in finite groups. These results seem 
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intuitive, but use material surprisingly far from group theory. By a reduced 
algebraic group scheme over Z, we mean (naively) that the group G is de- 
fined, as a subgroup of GL n (Z), with a finite number of polynomials over 
Z and that Z[G] has no nilpotents except 0. This is the case for the sim- 
ple algebraic groups, as well as their parabolic subgroups and generic MRR 
subgroups (see [9] or [15]). Such a group scheme has a group of points over 
every field. For an algebraically closed field k one may identify the group of 
points (of the group scheme) over k with the algebraic group (in the naive 
sense) over k. The field F p is the algebraic closure of the field of p elements 
for the prime p. Lawther [18] also uses some of these results. 

Proposition 4.1. Let G be a simple algebraic group scheme and X and P 
closed algebraic subgroup schemes ofG, all of which are reduced overly. For 
a field ¥ we denote by G(¥), X(¥) and P(F) the group of points over ¥ of 
G, X and P. Let k be an algebraically closed field. 

(i) // char k = then 

|X(k)\G(k)/P(k)| < oo limsup|X(F p )\G(Fp)/P(F p )| < oo. 

(ii) // char k = p > then 

|X(k)\G(k)/P(k)| < oo <j=> \X(¥ p )\G(¥ p )/P(¥ p )\ < oo. 

Proof. Part (ii) is proven in [12]. (We view the group X(k) x P(k) as acting 
on the affine space G(k). The assumption in [12] that X(k) x P(k) should 
be reductive is not used.) It may also be proven using a model theoretic 
argument similar in nature to the one we give now for part (i). For basic 
facts about model theory we refer to the textbooks by Fried-Jarden [10] or 
Hodges [13]. 

For p equal to or a prime, let ACF p be the theory of algebraically 
closed fields of characteristic p. Then ACF p is a complete theory. 

For a field F we identify G(¥) as a set of matrices in GL n (F) using the 
defining polynomials over Z. We make similar identifications for X and P. 
Since G, X and P are defined over Z we can express membership in G(¥), 
X(¥) and P(F) with first order sentences. Let <p be the sentence which, 
applied to the model F, gives 3 gi , . . . , g n € G(¥), Vg G G(F), 3x G X(F), 
3y G P(F), 3i G {1, . . . , n} such that xgy = gi. In other words, ip applied to 
F states that \X(¥)\G(¥) / P(¥)\ < n. 

Suppose X(k)\G(k)/P(k) is infinite in characteristic zero. Then <p is 
false in k. Then ACFq I 192 by completeness. This means that we have 
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a finite number of steps each of which only uses a finite number of axioms 
to derive —xp. In particular, only finitely many axioms which assert that 
m ■ 1 7^ are used and so there exists a prime po which is greater than every 
m which is used in this manner. For all primes p > po the axioms and steps 

which are used in the proof of ACFq I up may also be used to conclude 

ACF P h -up. Therefore, for all such p we have \X(¥ p )\G(¥ p ) / P(¥ p )\ > n 
whence limsup^^ \X(¥ p )\G(¥ p )/P(¥ p ) | > n. 

Conversely, a similar argument shows that ACFq h p ==> ACF P h <p> 
for all p sufficiently large. Therefore finiteness in characteristic implies 
boundedness of \X(¥ p )\G(¥ p )/P(¥ p )\ as p -» oo. □ 

Lemma 4.2. Zei G be a connected algebraic group defined over k = F p , 
o" : G — > G a Frobenius morphism, X and P closed subgroups which are 
a-stable. If X\G/P is infinite let C = 1. If X\G/P is finite let C be an 
upper bound on the number of components of stabilizers of X x P acting on 
G. Then 

— limsup \X (T n\G (T n / P a n I < \X\G/P\ < lim SUp \ X(jn\G(jri J Pfjn | . 
Proof. Suppose limsup \X a n\G a ™ / P a n \ is finite and less than m. We will 

n— »oo 

show that |X\G/P| < m. Let gi,...,g m G G. There is a number n € 
N such that gi,...,g m £ G CT n and m > \X (T n\G (T n /P^ \. Then at least 
two of <7i, . . . , g m are in the same X a n x P a n -orbit, whence they are in the 
same X x P-orbit. Since this holds for every g±, . . . ,g m £ G we see that 
\X\G/P\ < m. 

Suppose now that X\G/P is finite, let n be given and let (X\G/P) a n be 
the collection of <r n -stable X x P-orbits. Then the Lang-Steinberg Theorem 
[25] shows that G \X\G/P\ > C \(X\G/P) a n \ > \X a n\G a n /P a „ |. □ 

5 Character Theory and the Dimension Criterion 

Strategy and conventions. By Lemma 3.2 we may assume that G has 
simply connected derived subgroup when convenient and use Lemma 3.3, 
Lemma 3.4 etc. We wish to establish a criterion for infiniteness which is 
independent of characteristic. By section 4 it suffices to work over the al- 
gebraic closures of finite fields and establish infiniteness independently of 
the field. Let G be defined over the algebraic closure of a field of positive 
characteristic. Let a : G — > G be a q th power Frobenius morphism. We 
assume that X and P are cr-stable. Then to prove infiniteness it suffices to 
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show that \X a n\G a n / P a n I is unbounded as n approaches infinity. For fixed 
points we will use the notation G a , P s , etc as described in section 3. If H 
is a group, the notation [h] C H means h is an element of H and [h] is its 
.ff-conjugacy class. An element denoted by s will be semisimple, and an 
element denoted by u will be unipotent. This development roughly follows 
Lawther [18]. Essentially the following lemma regroups the terms in the 
inner product of characters. 

Lemma 5.1. Let X and P be subgroups of G with P parabolic. Define an 
equivalence relation on semisimple elements in X a as follows: s and t are 
equivalent ifG a)S and G a ,t are X a -conjugate. Denote the equivalence class of 
s by E(s,a). Choose a set S a of representatives of these equivalence classes. 
Then 

IW.I=E E nTT^i^H 1 ^)- 



Proof. Basic character theory gives 

\x a \G a /P a \ = (i£,i?;) Ga = (ix CT ,i&)x CT = E lG p»- 

Applying the Jordan-Chevalley decomposition within the finite group X a 
we get that this last sum is equal to 

toE E >&(»)■ 

sex,? mgx CTiS 
Now we claim that t € E(s, a) implies that 

E 1 &M= E 

Let x € X a with (G at t) x = Ga,s- The crucial step is to show that for all 
u G X a<t we have lp°(tu) = lp°(su x ). Once this is done, conjugation by x 
shows that the sums are equal. We work at the level of algebraic groups. 
Given u G X at , let (GIP) tu and (G/P) su x be the varieties of conjugates of 
P which contain tu and su x respectively. Then, by Lemma 3.6 l p ^(tu) and 
lp* 7 (su x ) are the numbers of a-rational points on these varieties. Let g G G 
such that t G P g , let T be a maximal torus of P 9 which contains t. We 
apply Lemma 3.4 to see that {Gt) x = G s . We have the following: 

T <G t T x <G S seT x =^ s G P gx . 
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It is now easy to see that tu G P 9 su x G P gx . Therefore, conjugation 

by x gives a a-equivariant bijection (G/P)t u —* {G/P) su x. Taking a-fixed 
points and applying Lemma 3.6 finishes the claim. 
Using the claim we have 

i^jE E 1 &M = j^E E \E(s,a)\i G p: (su). 

a sex,? ueXc^s CT| seS a ueXc^s 

To finish the proof note that if u and v! are conjugate in X a>s then lpj (su) = 
lp a (su'). Therefore the sum over unipotent elements in X CTjS may be re- 
placed by representatives of unipotent classes. The size of such a unipotent 
class is 



u 



\-^-a,s,u I 



□ 

Lemma 5.2. Let X and P be subgroups of G with X maximal rank. Let T 
be a a -stable maximal torus in X, W = Nq(T) the Weyl group, and s G T a . 
Let Z(s,a) = {t G G a | G ff , t = G a , s }, Z{s) = {t G G \ G t = G s } (as in 
Lemma 3.3) and let E(s, a) be as in the previous lemma. Then we have: 

(i) Z(s,a) = Z(s) a . 

(ii) ^\s x ° x Z(s,a)\ < \E(s,a)\ < \s x ° x Z(s, a) \ . 

Proof. Part (i). If t G Z{s) CT then G s = Gt which implies G ajS = G a j, 
whence Z(s) a C Z(s,a). If t G Z(s,a) then G a ^ s = G a ^- By Lemma 3.4 
this implies G s = Gt whence t G Z(s). 

Part (ii). It suffices to show that the following is a surjective map, with 
fibers bounded in size by \W\: 

(p : s x,T x Z(s, a) > E(s, a) . 

(s x ,t) i ► t x 

Note that this map is well-defined as every element in X a which centralizes 
s also centralizes t. To see that the map is surjective, let t G E(s,a) and 
let x G X(j with (G a ,t) x = G a , s - Then (s x ,t x ) is in the domain of (p and 
^(s x ~\t x ) =t. 

The remainder of the proof bounds the size of the fiber. Let (s x ,t±) be 
an element of the domain. We claim that 



(tf) = {(s w - lx ,tT) | w G W) f| (s*° x Z(s,a)). 
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It is easy to see that the set on the right is contained in (p 1 (tf). Conversely, 
let (s y ,t 2 ) e f^itf) and note t 2 = . 

Claim: T contains t\, t 2 = t xy , s, and s yx \ By assumption s £ T. 
Note that Lemmas 3.3 and 3.4 show Z(G a ^ s ) < T. By definition t\,t 2 € 
Z{G a ,s)- The following calculation shows that s yx € Z{G ayS ) (which fin- 
ishes the claim): 

G a ^ syx -i = (G a;S ) yx = (G a j 2 ) yx = G^yx-i = G a;tl = Go-, s . 

We will use [8, 3.7.1], and the (standard) notation which appears there. 
Write xy^ 1 = utwu' in the Bruhat canonical form. Since t 2 = t xy we 
have t 2 = tf. Since xy^ 1 conjugates s yx to s we have (s yx ) w = s and 
s y = s w- l x_ Therefore (s y ,t 2 ) = (s w ~ lx ,tf). □ 

Corollary 5.3. Let X be a maximal rank subgroup of G and P a parabolic 
subgroup of G. Let S a and Z(s, a) be as in Lemmas 5.1 and 5.2 respectively. 
We have 

^Ef14H * \XAG„/P a \ < E^fi&M' 

where each sum is taken over s € S a , [u] C X ajS . 

Proof. Combine Lemma 5.1 and the bounds for E(s,a) just obtained in 
Lemma 5.2. Note that 

\s x ° x Z(s,a)\ \X a , s \ = \Z(s,a)\ 

□ 

Proof 5.4 (Proof of the Dimension Criterion). We have that X is 
maximal rank, P parabolic and s € XnP. We assume dim Z(G s )+dim G s > 
dim X s + dim P s . We also assume that X and P are a stable where a is a 
standard q th power Frobenius map. 

Let G s , X s and P s denote the quotients of G s , X s and P s by Z(G S ). The 
dimension inequality we have assumed implies that dimG s — dim Z(G S ) > 
dimX s - dim Z(G S ) + dimP s - dimZ(G s ). This implies \X^\G^/P^\ = oo 
whence \X S \G S /P S \ = oo by Lemma 3.2. 

It remains to show that \X\G/P\ = oo. Using Corollary 5.3 and The- 
orem 4.2, it suffices to show that the term in Corollary 5.3 corresponding 
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to s € S a , 1 = [u] C X CTiS is unbounded as we replace cr with cr n and let 
n — > oo. This term is 

IG n 

It is easy to show that l p CT ™ (s) > g ' s whence this term is bounded below 

I -Per", s | 

by 

1 \Z(s,a n )\ IGUJ 



\W\ \X a n J \P a n J 



Therefore it suffices to show that 



1 \Z(s,a n )\\G a n, s \ 
Inn sup-— — -— r = oo. 

n-»oo | W | |Ao-n )S | \f a 



cr",S | 



By Lemmas 3.7 and 3.8 we have 

1 \Z(s,a n )\ \G a n s \ , ( g n)dimZ(G s )+dimG s 
hm sup — — — — j— — — '—r = hm C - 



\W\ \X a n }S \ \P a n,s\ ^oc ( g r i )dimX s +dimP s ' 

for some constant C. It is now easy to see that this limit is infinite. □ 



6 Proof of Theorems 1.2 and 1.3 

Throughout this section G is a simple algebraic group, X a generic MRR 
subgroup, P a parabolic subgroup with Levi factor L. Starting with Propo- 
sition 6.4 we will use H for arguments which apply to both X and L. 

Lemma 6.1. Let s G X n L. If either of the following holds then X\G/P 
is infinite: 

(i) G s is of type Ai and X s and L s are tori. 

(ii) G s is of type B>2, X s is a torus and L s is of type A\ or a torus. 
Proof. Using the dimension criterion it suffices to show that 

1 1 

dim Z(G S ) + - dim G s — dim X s — — dim L s > 0. 

It is easy to check in each case that the quantity on the left is at least 1 . □ 
Corollary 6.2. // either of the following hold then X\G/P is infinite: 
(i) X and L have conjugate Ai complements. 
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(ii) X has a B2 complement K , and for some conjugate K = K 9 we have 
that K (~l L is a MRR subgroup which is a torus or of type A\ . 

Proof. If G has rank 2 and (i) or (ii) holds then it is easy to show that 
X\G/P is infinite by dimension. 

Assume now that the rank of G is at least 3. If (i) holds let K be the 
A2 complement of X. Using Lemma 3.2 we replace P, if necessary, by a 
conjugate so that in (i) the A2 complements of X and L coincide, or so that 
in (ii), we may take K = K. Since the rank of G is at least 3, we have that 
K is a Levi factor of a parabolic (see Lemma 3.1), whence is of the form 
C G {S) for some torus S. Apply Lemma 3.3, to see that there exists s € S 
with the centralizer of s in G equal to K. We are done by the previous 
lemma. □ 

Corollary 6.3. If X has an A2 or B2 complement then X is not spherical. 

Proof. Apply Lemma 6.2, noting that the Levi factor for a Borel subgroup 
is a torus, which has every type of complement possible. □ 

Proposition 6.4. Let H be a generic MRR subgroup of G which does not 
appear in table 1. The following hold, and, in particular, H has an A2 or 
B2 complement in all cases. 

(i) If G has single root length, then H has an A2 complement. 

(ii) If H is the Levi factor of a parabolic with non-abelian unipotent radical 
then H has an A2, B2 or G2 complement. 

(hi) Let G equal B n or C n . 

(a) If n = 2 then H has a B2 complement. 

(b) If G = B n and H equals D n -\T\ or D ni D n2 then H has a B2 
complement. 

(c) Otherwise H has an A2 complement. 

(iv) Let G = B n . If H is a Levi factor then there exists a MRR subgroup 
K of type B2 with H n K a MRR subgroup which is either a torus or 
of type A 1 . 

(v) If G = -F4 the maximal possibilities for H are C3T1, A2A2, B3T1, 
A\A\B2, A\A^, D4. The first possibility has a long A2 complement, 
the next has both long and short A2 complements, and the rest have 
short A2 complements. In particular, if L is a Levi factor for a 
parabolic subgroup which is not an end node parabolic, then L has 
both long and short A2 complements. 

The proof of this proposition is delayed until the next section. 
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Proof 6.5 (Proof of Theorem 1.2). The work of Brundan [6] and The- 
orem 2.2 show that (ii) ==> (i). Corollary 6.3 shows that (i) (iii). 
Proposition 6.4 shows that (iii) (ii). □ 

Proof 6.6 (Proof of Theorem 1.3). We assume that X and L are not 

spherical we will show that X\G/P is infinite. 

If G = C?2 then by dimension one finds that X non-spherical implies 
X\G/P is infinite. For the remainder of the proof assume G / G2. 

Recall our convention that D± is a 1-dimensional torus. If (G, X) 7^ 
(B n , D ni D n2 ) then let Hx be an A2 complement for X and let Hi be an A2 
complement of L, of the same length as Hx (length is only an issue for F4). 
If G = B2, or (G,X) = (B n , D ril D n2 ) then let Hx be a B2 complement for 
X and let Hi be a MRR subgroup of type B2 with L n Hi a MRR subgroup 
of type A\ or a maximal torus. Apply Lemma 3.1 to see that Hx and Hi 
are conjugate. Apply Lemma 6.2 to see that X\G/P is infinite. □ 

We offer some insight into this result. Suppose the rank of G is at least 
3 and that (G, X) ^ (B n , D ni D n2 ). If X and L are both not spherical, then 
both have A2 complements. Assume these A2 complements are conjugate. 
Let T2 and B be a maximal torus and Borel subgroup of A2 respectively. 
Then T2\A2/B is infinite, and, in a sense, it may be "embedded" in X\G/P. 
The embedding comes from the expansion of terms in the inner product 
of finite group characters corresponding to X\G/P. Take s such that its 
centralizer (modulo a central torus) gives T2, A2 and B in place of X, G 
and P. The terms which arise by applying Corollary 5.3 to X, G and P 
and s are the same terms one would obtain by applying Corollary 5.3 to the 
groups X = T 2 ,G = A 2 and P = B. 

7 Proof of Proposition 6.4 

We prove parts (i) and (ii) immediately. Parts (iii)-(v) follow after Corollary 
7.3. 

Proposition 6.4(i). Recall G has single root length and H is a MRR 
subgroup which fails to appear in table 1. Then, by Theorem 2.2, H is not 
anti-open, whence there exist a, (3, a + (3 G &(G) — Q(H). Let T be the 
maximal torus used to define these roots and denote the corresponding root 
groups by U a , Up etc. Let K be the group generated by T and U± a , U±p. 
Then K is an A2 complement for G. 

Proposition 6.4(h). Recall that H is the Levi factor of a parabolic 
with non-abelian unipotent radical. Let Q be the unipotent radical of the 
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parabolic associated with H. Fix a maximal torus T in H and let U a and 
Up be root groups contained in Q which do not commute. Let K be the 
group generated by T and U± a , U±p. Then K is of type A2, B2 or G2 and it 
remains to show that K n H = T. Let Q~ be the opposite unipotent radical 
of Q. Note that &(K) < U $(Q~) whence n = whence 

K (~) H = T. Thus if is an A 2 , £ 2 or G 2 complement for F. 

Lemma 7.1. Xei 99 6e an irreducible root system in a Euclidean space E 
wii/i inner product ( , ). Zei <p be a proper, closed subsystem of if. Then 
ip — (p spans E and for each (3 G (p there exists a G p — (p with (a, (3) 7^ 0. 

Proof. Let n be the dimension of E and fix a Dynkin diagram A of p. Given 
a, (3 G A the path connecting a to (3 is the shortest such path and includes 
a and (3. The sum over this path means the sum of each element of A which 
is contained in the path. It is easy to check that such a sum is itself a root. 

For the first conclusion it suffices to show that we have n independent 
vectors in p — p. Since <p is a proper, closed subsystem we have that A — <p is 
non-empty. For each element a G A let j a be the path connecting a to some 
element of A — (p. We re- index these paths so that for i G {1, . . . , n} the path 
7i contains a node which does not appear in 71, ... , 7^-1 . Let f3\, . . . , (3 n be 
the sums over the paths just constructed. By the manner in which the paths 
ji were indexed, it is easy to see that (3\, . . . , (3 n are linearly independent. 
By the manner in which the paths were chosen, we may write each (3i as the 
sum of a root in (p and a root outside of (p. This shows that is not in (p. 

For the final conclusion note that f3 is not orthogonal to E, whence it is 
not orthogonal to <p — (p. □ 

Corollary 7.2. Let G be a reductive algebraic group, H a MRR subgroup. 
Let ip < 3>(G) be an irreducible root system and let <p(H) = <p fl $(H). 
Suppose p is a closed subsystem of p with ip > p > p{H). 

(i) If ip has single root length then H has an A2 complement (whose length 
is the same as ip). 

(ii) // p is closed, G = B n and <p — p(H) contains a short root then H has 
a B2 complement. 

Proof. Fix (3 G <p — <p{H) and assume (3 is short if (ii) holds. By the previous 
lemma there exists a G <p — <p with (a, (3) 7^ 0. Note that a 7^ ±/3 and that 
ia + j(3 G 3>(G) ia + j(3 G p. If (a, (3) > we may replace one root 

with its negative and assume that (a, [3) < 0, whence a + (3 G p>. Since 
a G" <p and (3 G p we see that a + (3 G" p. Similarly, we see that a + 2(3 G" (p 
(of course it may not even be a root) and that 2a + (3 is not a root. Let T 
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be a maximal torus in H used to define these roots. Set K to be the group 
generated by T and all root groups Ui a+ jp where ia + j(3 is a root with i 
and j integers. Then KC\H = T and K is of type A2, -B2 or G<i- If (i) holds 
then K is a group of type Ai- If (h) holds then K is a group of type Bi 
since [3 is short and B n has no MRR subgroups of type short A2 or G2. □ 

Corollary 7.3. Let G be a reductive algebraic group, H a MRR subgroup, 
ip < an irreducible root system with single root length. If ip(H) is 

submaximal in <p then there exists <p as in the previous corollary. 

Proof. In a root system with single root length, every root subsystem is 
closed. □ 

Proof 7.4 (Proof of Proposition 6.4 (iii),(iv),(v)). Part (hi). Recall 
G equals B n or C n and H is a generic MRR subgroup which does not appear 
in table 1. 

Part (a). Since n = 2 and H does not appear in table 1 we see that H 
is just a torus and G itself is a B2 complement. 

Part (b). We have n > 3, G = B n and H € {D n _iTi, D ni D n2 }. If 
H = D ni D n2 then m > 2. Let <p = $(G) so p(fl') = $(#). If H = D n - X T X 
let £ = 3>(i? n _iTi). If = D ni D n2 let £ = 3>(i? ni Z) n2 ). In each case ^ 
contains a short root and <p(H) does not. Then (p — <p(H) contains a short 
root and we are done by part Corollary 7.2(h). 

Part (c). Note that n > 3 and if G = B n that H {Z? n _iTi, D ni D n2 }. 
If G = B n let ip and <p(H) equal the long roots in $(G) and $(-&) respec- 
tively. If G = C n let if and <p(i?) equal the short roots in $>(G) and $>{H) 
respectively. In both cases (p is of type D n , and is irreducible since n > 3. 
The maximal subsystems of (/? are A n -i, D n _i and D ni D n2 . The subsystem 
99(7/) cannot equal L> n , D„_i or D ni D n2 as this would contradict 

either the assumption that if is not in table 1 or the extra restrictions on 
H when G = B n . Therefore <p(H) is a submaximal subsystem of ip and we 
are done by Corollary 7.3. 

Part (iv). We have that G = B n and that H is a Levi factor of G. Let T 
be a maximal torus in H, let ai, . . . , a n be the nodes in the Dynkin diagram 
of G in the usual order and suppose that H is described by "crossing off" 
certain nodes. Let j3\ = ct\ and fa = «2 + ■ ■ ■ + ct n . Let K be the group 
generated by T, U±p 1 , and U±p 2 . Then K is of type B^. The subsystem 
&(H) may contain either (3\ or /32, but not both. Similarly &{H) may not 
contain i(3\ + with % and j positive integers. This shows that K D H is 
either T or of type A\. 
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Part (v). We have that G = F4. To construct all short A2 complements, 
take ip equal to all the short roots in <f>(Fi), so tp(H) equals all the short 
roots in <&(H). Observe that ip is of type D4. By examining each possibility 
for H it is easy to verify that f(H) is submaximal in a D4 root system and 
we are done by Corollary 7.3. To construct the long A2 complements, one 
proceeds similarly with ip equal to all the long roots in $(^4). □ 
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